<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
                "http://www.w3.org/TR/REC-html40/loose.dtd">
<html>
<head>
  <title>Description of glmhess</title>
  <meta name="keywords" content="glmhess">
  <meta name="description" content="GLMHESS Evaluate the Hessian matrix for a generalised linear model.">
  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
  <meta name="generator" content="m2html &copy; 2003 Guillaume Flandin">
  <meta name="robots" content="index, follow">
  <link type="text/css" rel="stylesheet" href="../../m2html.css">
</head>
<body>
<a name="_top"></a>
<div><a href="../../menu.html">Home</a> &gt;  <a href="#">ReBEL-0.2.7</a> &gt; <a href="#">netlab</a> &gt; glmhess.m</div>

<!--<table width="100%"><tr><td align="left"><a href="../../menu.html"><img alt="<" border="0" src="../../left.png">&nbsp;Master index</a></td>
<td align="right"><a href="menu.html">Index for .\ReBEL-0.2.7\netlab&nbsp;<img alt=">" border="0" src="../../right.png"></a></td></tr></table>-->

<h1>glmhess
</h1>

<h2><a name="_name"></a>PURPOSE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>GLMHESS Evaluate the Hessian matrix for a generalised linear model.</strong></div>

<h2><a name="_synopsis"></a>SYNOPSIS <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="box"><strong>function [h, hdata] = glmhess(net, x, t, hdata) </strong></div>

<h2><a name="_description"></a>DESCRIPTION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre class="comment">GLMHESS Evaluate the Hessian matrix for a generalised linear model.

    Description
    H = GLMHESS(NET, X, T) takes a GLM network data structure NET,   a
    matrix X of input values, and a matrix T of target values and returns
    the full Hessian matrix H corresponding to the second derivatives of
    the negative log posterior distribution, evaluated for the current
    weight and bias values as defined by NET. Note that the target data
    is not required in the calculation, but is included to make the
    interface uniform with NETHESS.  For linear and logistic outputs, the
    computation is very simple and is  done (in effect) in one line in
    GLMTRAIN.

    [H, HDATA] = GLMHESS(NET, X, T) returns both the Hessian matrix H and
    the contribution HDATA arising from the data dependent term in the
    Hessian.

    H = GLMHESS(NET, X, T, HDATA) takes a network data structure NET, a
    matrix X of input values, and a matrix T of  target values, together
    with the contribution HDATA arising from the data dependent term in
    the Hessian, and returns the full Hessian matrix H corresponding to
    the second derivatives of the negative log posterior distribution.
    This version saves computation time if HDATA has already been
    evaluated for the current weight and bias values.

    See also
    <a href="glm.html" class="code" title="function net = glm(nin, nout, outfunc, prior, beta)">GLM</a>, <a href="glmtrain.html" class="code" title="function [net, options] = glmtrain(net, options, x, t)">GLMTRAIN</a>, <a href="hesschek.html" class="code" title="function h = hesschek(net, x, t)">HESSCHEK</a>, <a href="nethess.html" class="code" title="function [h, varargout] = nethess(w, net, x, t, varargin)">NETHESS</a></pre></div>

<!-- crossreference -->
<h2><a name="_cross"></a>CROSS-REFERENCE INFORMATION <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
This function calls:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="consist.html" class="code" title="function errstring = consist(model, type, inputs, outputs)">consist</a>	CONSIST Check that arguments are consistent.</li><li><a href="glmfwd.html" class="code" title="function [y, a] = glmfwd(net, x)">glmfwd</a>	GLMFWD	Forward propagation through generalized linear model.</li><li><a href="hbayes.html" class="code" title="function [h, hdata] = hbayes(net, hdata)">hbayes</a>	HBAYES	Evaluate Hessian of Bayesian error function for network.</li></ul>
This function is called by:
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="glmtrain.html" class="code" title="function [net, options] = glmtrain(net, options, x, t)">glmtrain</a>	GLMTRAIN Specialised training of generalized linear model</li></ul>
<!-- crossreference -->

<h2><a name="_subfunctions"></a>SUBFUNCTIONS <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<ul style="list-style-image:url(../../matlabicon.gif)">
<li><a href="#_sub1" class="code">function hdata = rearrange_hess(net, j, out_hess, hdata)</a></li></ul>
<h2><a name="_source"></a>SOURCE CODE <a href="#_top"><img alt="^" border="0" src="../../up.png"></a></h2>
<div class="fragment"><pre>0001 <a name="_sub0" href="#_subfunctions" class="code">function [h, hdata] = glmhess(net, x, t, hdata)</a>
0002 <span class="comment">%GLMHESS Evaluate the Hessian matrix for a generalised linear model.</span>
0003 <span class="comment">%</span>
0004 <span class="comment">%    Description</span>
0005 <span class="comment">%    H = GLMHESS(NET, X, T) takes a GLM network data structure NET,   a</span>
0006 <span class="comment">%    matrix X of input values, and a matrix T of target values and returns</span>
0007 <span class="comment">%    the full Hessian matrix H corresponding to the second derivatives of</span>
0008 <span class="comment">%    the negative log posterior distribution, evaluated for the current</span>
0009 <span class="comment">%    weight and bias values as defined by NET. Note that the target data</span>
0010 <span class="comment">%    is not required in the calculation, but is included to make the</span>
0011 <span class="comment">%    interface uniform with NETHESS.  For linear and logistic outputs, the</span>
0012 <span class="comment">%    computation is very simple and is  done (in effect) in one line in</span>
0013 <span class="comment">%    GLMTRAIN.</span>
0014 <span class="comment">%</span>
0015 <span class="comment">%    [H, HDATA] = GLMHESS(NET, X, T) returns both the Hessian matrix H and</span>
0016 <span class="comment">%    the contribution HDATA arising from the data dependent term in the</span>
0017 <span class="comment">%    Hessian.</span>
0018 <span class="comment">%</span>
0019 <span class="comment">%    H = GLMHESS(NET, X, T, HDATA) takes a network data structure NET, a</span>
0020 <span class="comment">%    matrix X of input values, and a matrix T of  target values, together</span>
0021 <span class="comment">%    with the contribution HDATA arising from the data dependent term in</span>
0022 <span class="comment">%    the Hessian, and returns the full Hessian matrix H corresponding to</span>
0023 <span class="comment">%    the second derivatives of the negative log posterior distribution.</span>
0024 <span class="comment">%    This version saves computation time if HDATA has already been</span>
0025 <span class="comment">%    evaluated for the current weight and bias values.</span>
0026 <span class="comment">%</span>
0027 <span class="comment">%    See also</span>
0028 <span class="comment">%    GLM, GLMTRAIN, HESSCHEK, NETHESS</span>
0029 <span class="comment">%</span>
0030 
0031 <span class="comment">%    Copyright (c) Ian T Nabney (1996-2001)</span>
0032 
0033 <span class="comment">% Check arguments for consistency</span>
0034 errstring = <a href="consist.html" class="code" title="function errstring = consist(model, type, inputs, outputs)">consist</a>(net, <span class="string">'glm'</span>, x, t);
0035 <span class="keyword">if</span> ~isempty(errstring);
0036   error(errstring);
0037 <span class="keyword">end</span>
0038 
0039 ndata = size(x, 1);
0040 nparams = net.nwts;
0041 nout = net.nout;
0042 p = <a href="glmfwd.html" class="code" title="function [y, a] = glmfwd(net, x)">glmfwd</a>(net, x);
0043 inputs = [x ones(ndata, 1)];
0044 
0045 <span class="keyword">if</span> nargin == 3
0046    hdata = zeros(nparams);    <span class="comment">% Full Hessian matrix</span>
0047    <span class="comment">% Calculate data component of Hessian</span>
0048    <span class="keyword">switch</span> net.outfn
0049 
0050    <span class="keyword">case</span> <span class="string">'linear'</span>
0051       <span class="comment">% No weighting function here</span>
0052       out_hess = [x ones(ndata, 1)]'*[x ones(ndata, 1)];
0053       <span class="keyword">for</span> j = 1:nout
0054          hdata = <a href="#_sub1" class="code" title="subfunction hdata = rearrange_hess(net, j, out_hess, hdata)">rearrange_hess</a>(net, j, out_hess, hdata);
0055       <span class="keyword">end</span>
0056    <span class="keyword">case</span> <span class="string">'logistic'</span>
0057       <span class="comment">% Each output is independent</span>
0058       e = ones(1, net.nin+1);
0059       link_deriv = p.*(1-p);
0060       out_hess = zeros(net.nin+1);
0061       <span class="keyword">for</span> j = 1:nout
0062          inputs = [x ones(ndata, 1)].*(sqrt(link_deriv(:,j))*e);
0063          out_hess = inputs'*inputs;   <span class="comment">% Hessian for this output</span>
0064          hdata = <a href="#_sub1" class="code" title="subfunction hdata = rearrange_hess(net, j, out_hess, hdata)">rearrange_hess</a>(net, j, out_hess, hdata);
0065       <span class="keyword">end</span>
0066       
0067    <span class="keyword">case</span> <span class="string">'softmax'</span>
0068       bb_start = nparams - nout + 1;    <span class="comment">% Start of bias weights block</span>
0069       ex_hess = zeros(nparams);    <span class="comment">% Contribution to Hessian from single example</span>
0070       <span class="keyword">for</span> m = 1:ndata
0071          X = x(m,:)'*x(m,:);
0072          a = diag(p(m,:))-((p(m,:)')*p(m,:));
0073          ex_hess(1:nparams-nout,1:nparams-nout) = kron(a, X);
0074          ex_hess(bb_start:nparams, bb_start:nparams) = a.*ones(net.nout, net.nout);
0075          temp = kron(a, x(m,:));
0076          ex_hess(bb_start:nparams, 1:nparams-nout) = temp;
0077          ex_hess(1:nparams-nout, bb_start:nparams) = temp';
0078          hdata = hdata + ex_hess;
0079       <span class="keyword">end</span>
0080     <span class="keyword">otherwise</span>
0081       error([<span class="string">'Unknown activation function '</span>, net.outfn]);
0082     <span class="keyword">end</span>
0083 <span class="keyword">end</span>
0084 
0085 [h, hdata] = <a href="hbayes.html" class="code" title="function [h, hdata] = hbayes(net, hdata)">hbayes</a>(net, hdata);
0086 
0087 <a name="_sub1" href="#_subfunctions" class="code">function hdata = rearrange_hess(net, j, out_hess, hdata)</a>
0088 
0089 <span class="comment">% Because all the biases come after all the input weights,</span>
0090 <span class="comment">% we have to rearrange the blocks that make up the network Hessian.</span>
0091 <span class="comment">% This function assumes that we are on the jth output and that all outputs</span>
0092 <span class="comment">% are independent.</span>
0093 
0094 bb_start = net.nwts - net.nout + 1;    <span class="comment">% Start of bias weights block</span>
0095 ob_start = 1+(j-1)*net.nin;     <span class="comment">% Start of weight block for jth output</span>
0096 ob_end = j*net.nin;             <span class="comment">% End of weight block for jth output</span>
0097 b_index = bb_start+(j-1);       <span class="comment">% Index of bias weight</span>
0098 <span class="comment">% Put input weight block in right place</span>
0099 hdata(ob_start:ob_end, ob_start:ob_end) = out_hess(1:net.nin, 1:net.nin);
0100 <span class="comment">% Put second derivative of bias weight in right place</span>
0101 hdata(b_index, b_index) = out_hess(net.nin+1, net.nin+1);
0102 <span class="comment">% Put cross terms (input weight v bias weight) in right place</span>
0103 hdata(b_index, ob_start:ob_end) = out_hess(net.nin+1,1:net.nin);
0104 hdata(ob_start:ob_end, b_index) = out_hess(1:net.nin, net.nin+1);
0105 
0106 <span class="keyword">return</span></pre></div>
<hr><address>Generated on Tue 26-Sep-2006 10:36:21 by <strong><a href="http://www.artefact.tk/software/matlab/m2html/">m2html</a></strong> &copy; 2003</address>
</body>
</html>